See above: what are we expecting users to set the u64 to?

Shouldn't we rather use a dedicated PaymentMetadata type for this, or even an Option<Box<dyn Writeable>> or similar?

Answered at #4584 (comment) but I do really want to force users into the K-V-map box here. Can do a type if we want but its a bit awkward that we have a separate BOLT 11 and BOLT 12 PaymentMetadata type...

The Box<dyn Writeable> would get a bit ugly at read time. You'd need to parameterize the onion message parsing with your types.

Why use a map? Wouldn't a user's wrapper struct be just as efficient encoding -- or could be even more efficient if each part is of a fixed length? Deserialization would be simpler, too, if they didn't need to iterate a map.

The intention is both to force users to think about forward compatibility from day one (in the sense that its already a K-V) but also because we want to reserve some keys in LDK (eg in #4463).

The intention is both to force users to think about forward compatibility from day one (in the sense that its already a K-V)

Isn't that what using our serialization macros provide? Blinded paths are also not expected to be useable indefinitely, so the compatibility argument isn't as strong, IMO.

FWIW, the rationale given in the commit message only mentions composability.

but also because we want to reserve some keys in LDK (eg in #4463).

Wouldn't it be simpler to add a dedicated field for anything LDK-specific? Then we wouldn't need custom parsing / serialization logic (for each variant of PaymentContext) or need to worry about checking if users took a reserved type. This would need to be added somewhere in the code vs a simple declaration in the impl_writeable_tlv_based uses.

Isn't that what using our serialization macros provide? Blinded paths are also not expected to be useable indefinitely, so the compatibility argument isn't as strong, IMO.

Right, for cases where there's a single top-level struct which the downstream code wants to use.

Wouldn't it be simpler to add a dedicated field for anything LDK-specific?

Yea, this is probably true for #4463, but I'm not sure how this would work, eg, if LDK Node had something it wanted to add? Would we add a field here just for LDK Node's usage? Or would LDK Node define a struct which then had a similar user-defined sub-struct?

I guess its not clear to me how exactly a struct would work, anyway, would we parameterize ~everything by the custom context struct?

Right, for cases where there's a single top-level struct which the downstream code wants to use.

We could advise to include a top-level struct with a field holding a sub-struct with their data. Then they would simply add a new struct / field for another use case. Alternatively, their serialization could have a version number if they ever want to swap-out a top-level struct.

Yea, this is probably true for #4463, but I'm not sure how this would work, eg, if LDK Node had something it wanted to add? Would we add a field here just for LDK Node's usage? Or would LDK Node define a struct which then had a similar user-defined sub-struct?

Right, I guess we'd need to know how LDK Node would communicate to LDK to set the field and how it can be communicated back to LDK Node when handling a message sent over the blinded path. For payment paths, we'd expose it through PaymentClaimable. But how will it be set?

For the LSP case, I thought the idea would be to communicate data through the blinded message path. And then LDK would have logic for setting data in the blinded payment path when handing an invoice request containing that data.

For other use cases, we'd probably force them to use OffersMessageFlow (or a different system / handler if this is not offers-related blinded path usage). We could make LDK Node use OffersMessageFlow, too. Or just provide the tighter integration with LDK as described above.

I guess its not clear to me how exactly a struct would work, anyway, would we parameterize ~everything by the custom context struct?

I was saying to just have a Vec<u8> and allow consumers to parse as needed. They could use a custom struct and parameterize everything with it. But for LDK-specific data, we'd have our own dedicated struct / field.

I was saying to just have a Vec and allow consumers to parse as needed. They could use a custom struct and parameterize everything with it. But for LDK-specific data, we'd have our own dedicated struct / field.

Ah, yea, I really don't love that. At the cost of two extra bytes forcing them to have a specified key effectively forces them to have forward/backwards compat (which, indeed, doesn't matter as much in blinded payment paths but we're also using the same field in offers blinded paths, which are long-lived) and allows user data to live parallel with LDK Node (or other LDK non-lightning-crate) data. It seems very much worth it to have that and I don't really think the API complexity cost is all that high here.